Gradients should stay on path: better estimators of the reverse- and forward KL divergence for normalizing flows

Abstract We show how to use the path-wise derivative estimator for both forward reverse Kullback–Leibler divergence any practically invertible normalizing flow. The resulting path-gradient estimators are straightforward implement, have lower variance, and lead not only faster convergence of training but also better overall approximation results compared standard total gradient estimators. demonstrate that is less susceptible mode-collapse. In light our results, we expect will become new method train flows variational inference.